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Abstract 

As we face increasing accountability in higher education, how we measure 
student learning should exceed the calls for an account of learning that 
places students at the center. Qualitative approaches to assessment and 
theoretical underpinnings gleaned from the qualitative research tradition 
may provide a way that we can support a more holistic view of the student 
college experience, and in some cases, provide a more comprehensive 
narrative than quantitative assessment methods alone. We argue that as 
student populations change rapidly, no longer will large, indirect measures of 
student learning stand on their own as a comprehensive explanation; instead, 
we turn to philosophical and epistemological foundations of qualitative 
inquiry as one way to think about capturing the complexity within student 
learning experiences. Qualitative approaches to assessment could provide 
new possibilities for our own knowledge in regards to assessment, and also 
provide a space in which we learn more—about learning. 


To Learn More about Learning: 
The Value-Added Role of Qualitative 
Approaches to Assessment 


iven the repeated clarion calls for increased accountability within higher 
education (most recently, Kuh, Jankowski, Ikenberry, & Kinzie, 2014), practitioners are 
under great pressure to report how and what students learn in college and the extent to 
which this learning will transfer outside of postsecondary contexts. An increasing number of 
interested parties ask those of us involved in higher education to make connections between 
student experiences inside and outside of the classroom, and how these experiences translate 
into student learning. 

For example, the 2006 Spellings Report discussed the assessment of student learning 
as one of the most important priorities for higher education in the future (Contreras-McGavin 
& Kezar, 2007). Responding to these calls responsibly and holistically would require a variety 
of measures to catalogue an understanding of student learning. The overarching goal within 
this accountability context, then, is to measure student learning in higher education. Or, 
put a bit differently, to somehow learn about learning. The irony is that in attempting to 
better understand a complex interaction such as learning, we often turn to data collection 
procedures that seek to reduce and/or control for complexity. In an attempt to produce 
objective findings, for example, we rely on particular methods, procedures, and criteria that 
together seek to limit personal judgment, speculation, critique, and interpretation. Indeed, 
CORRESPONDENCE we have come to rely on particular methods in themselves (Schwandt, 1996) as a way to 

produce reliable knowledge about outcomes for student learning. In this sense, methods 
Email may serve as a filter of sorts and in so doing, limit our ability to ask deep questions about the 
daniel.newhart@oregonstate.edu complex nature of student learning. 

Yet, we know that quantitative measures alone do not capture a holistic view of 
student learning. Student learning is complex (Keeling, 2006; Keeling, Wall, Underhile, & 
Dungy, 2008) and as such, it rarely happens that one type of measure is able to account for 
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learning in such a way as to create a coherent and comprehensive narrative. Moreover, the 
data alone do not tell us what to do or, as Dowd (2005) states, “The data don’t drive.” In the 
end, we are still confronted with the necessary tasks of interpretation and understanding and 
forwarding evidence-influenced recommendations, which is very important as well. 

The challenge we undertake in this essay is one of opportunity. Increasing calls for 
accountability provide us great space to shape the way we think about and measure student 
learning in postsecondary contexts. Drawing from the philosophical and epistemological 
tradition of qualitative inquiry, we argue that qualitative methods have much to add to what we 
know—and can know—about student learning in higher education. Specifically, we consider 
the value-added nature of qualitative approaches to assessment and then turn to direct and 
indirect measures of student learning as examples of engaging this work. Instrumental to our 
argument are the foundational principles of qualitative research, as well as how these principles 
can transfer to the area of qualitative approaches to assessment. 

Foundations of Qualitative Research 

The arena of qualitative inquiry is diverse. A variety of intellectual and disciplinary 
approaches to the study of social phenomena fall under the umbrella of qualitative research, as 
do contested beliefs about the nature of reality. Yet, while the dynamics within the field make 
qualitative inquiry difficult to define, there exist shared characteristics or assumptions that we 
might consider foundational. 

Qualitative research is not simply about interviews and focus groups, but about 
meaning. To examine the world qualitatively means to study and represent the meaning of a 
particular social interaction, such as learning in postsecondary contexts. Using explanation 
and description, qualitative researchers attempt to interpret social reality or understand the 
meaning of social action. They do so by studying things “.. .in their natural settings, attempting 
to make sense of, or interpret, phenomena in terms of the meanings people bring to them” 
(Denzin & Lincoln, 2008, p. 4). In this sense, qualitatively grounded approaches to social 
science research involve interactive processes among the researchers, the participants, and 
their socio-political contexts. Because social life is complex and layered, qualitative researchers 
examine social phenomena within this context and seek to account for multiple influences 
upon the meaning of social behavior. 

The goal of qualitative social science research, therefore, is not necessarily to arrive at 
a final answer about a particular social phenomenon, but instead to provide evidence towards 
a certain way of thinking about it. The assumption is that there is no one right answer that 
can be unearthed with the appropriate methods, but rather many credible answers that may 
provide insight into better understanding a particular issue. As Denzin and Lincoln (2008) 
argue, each qualitative project “makes the world visible in a different way” (p. 5) and in the 
context of understanding student learning, a collection of these approaches may provide a 
holistic picture of learning experiences in higher education. 

Our assumptions about learning, including what it means to learn and how one should 
learn, stem from larger conventions that we hold about reality. Invariably, these assumptions 
guide our understandings of how best to measure the social world and can limit our ability to 
capture complexity. For example, how do we imagine the world in order to study it? Likewise, 
how do we imagine student learning so that it can be assessed? In attempting to assess student 
learning with certainty, the complexity can be reduced to easily identifiable variables. Because 
learning does not occur in a vacuum and failing to learn is quite different than refusing to 
learn (Kohl, 1994), it is important that our methods capture the complexity of learning rather 
than controlling for it. Turning to the field of qualitative inquiry is useful because qualitative 
researchers challenge desires for absolute certainty. Instead, by understanding that methods 
serve as a filter for what we can discover and learn, qualitative researchers try to work through 
these complexities, or what some refer as the mess of doing social science research (Lather, 
2009; St. Pierre & Pillow, 2000), by examining how these complexities function within a 
particular context. 

To approach assessment qualitatively, then, is to seek complexity rather than suppress 
it. In the context of attempting to capture student learning, it means that we must begin by 
asking: What does it mean to learn? We can then, by extension, engage the most appropriate 
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approaches to understanding and measuring that learning. It also means that we design 
qualitative approaches that allow us to capture unanticipated data (Becker, 1996) by opening 
up opportunities for thinking about and assessing student learning differently. Thus, qualitative 
approaches to assessment provide us an opportunity to reconsider what it means to learn by 
placing ourselves in a position to learn more about learning. 

Accountability on Multiple Levels 

As higher education becomes more diverse in the constituents that it serves and its 
increasing reach into various sectors, it is important to account for student learning in such 
a way that the experiences students have after they leave our institutions can be tied to their 
experiences within our contexts. In a sense, we must be accountable for showing that the time, 
money, and effort that students spend in our institutions can translate into outside of college 
arenas and that this investment is worth the economic and personal cost of attending college. 
Given the increased calls for accountability, our measures must become more complex as well. 
In fact, this may already be occurring. According to recently released research (Kuh et al., 
2014), the number of rubrics used to assess student learning in higher education is increasing, 
indicating that surveys are perhaps being relied upon less in the measurement of learning—or 
perhaps that our narrative around student learning is becoming more complex. 

Qualitative approaches to assessment, whether used alone or in tandem with 
quantitative approaches, can help provide a deeper understanding of student learning. Since 
qualitative assessment may allow for more depth than quantitative assessment, we can begin 
to answer the calls for more accountability for the work we are doing in more detail—as 
well as respect the diverse student experiences that occur at our institutions. Our measures, 
through qualitative approaches to assessment, be they direct or indirect, allow us a way to 
be accountable to the calls for proof that learning is occurring at the university. Qualitative 
approaches to assessment also highlight some of what we have learned from the qualitative 
research tradition, which would espouse that qualitative methods allow for a more full 
description than solely looking at phenomena through a quantitative lens. 

The point here is not to disparage quantitative approaches to assessment, but rather 
reiterate that student learning is complex (Keeling, 2006; Keeling et al., 2008) and that, in 
some cases, multiple types of methods may be required to get the complete picture of student 
learning. To merely use quantitative methods may not be honoring the learning experiences 
of students, particularly if they are members of smaller groups who may not be able to be 
analyzed using more advanced statistical procedures that require larger sample sizes. As 
explored below, qualitative approaches to assessment may be better equipped to measure 
learning from those students who might not learn through the ways in which our measures, 
either direct or indirect, purport to measure their learning. As we seek to understand 
more about learning, we could engage this learning mismatch by using the teachings of the 
qualitative tradition, and avoid the possible pitfall of our methods acting as a filter that would 
not be able to measure this lack of learning. Our measures could help answer the call that 
asks us to show that students are learning in complex ways. But first, what does qualitative 
work have to add to the field of assessment? 

The Value-Added Role of Qualitative Approaches to Assessment 

First, in an effort to show the value-added role of qualitative assessment, one must look 
into the qualitative research literature. However, since assessment and research are sometimes 
considered separate processes (Schuh, 2009), one must engage this from a framework which 
considers assessment and research as possibly connected and symbiotic (Newhart, 2011). By 
connecting them in the modes of guidance, rather than the intended purpose of research or 
assessment (and keeping them as separate endeavors), one can learn from the rich qualitative 
research tradition, specifically in the areas of paradigms and methodology, as these may apply 
to both assessment and research. Considering research and assessment as linked can allow 
for a study to be conducted in such a way to not only research the phenomena of interest, but 
then upon implementation, assess it (Newhart, 2011). 

For example, an assessment to determine the efficacy of a particular teaching style 
in students seeking information from the library on a topic of interest for a class might 
involve a direct qualitative assessment measure. This might be an observational assessment 
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of students using the potentially learned strategies to find information in a database. However, 
we soon might find that one particular group of students is not responding to the teaching 
methods, due to some specific reason that we may not know about. This then emerges as 
a research question, as we might wish to investigate why this particular group of students 
is not responding to the teaching method—and how we might adapt our teaching style to 
include these students. We can then take the recommendations from the research and apply 
them, and assess whether those recommendations worked for this population, continuing 
the process of assessing and researching. 

Second, using the framework provided to us from qualitative research also allows us to 
critically analyze and evaluate qualitative approaches to assessment which purport to measure 
student learning. According to Reeling et al. (2008), qualitative approaches to assessment must 
be founded upon theory and work from a “sound theoretical base” (p. 47). The authors divide 
qualitative assessment into five traditional areas, specifically: “Biography, phenomenology, 
grounded theory, ethnography, and case study” (pp. 47-48). These areas are referenced 
elsewhere when speaking of qualitative research as methodologies (Jones, Torres, & Arminio, 
2014), and are considered that which can guide our inquiry intentionally. 

As a result of this, qualitative approaches to assessment are perhaps inaccurately 
thought of as taking more time to engage than quantitative modes of assessment. However, 
Harper and Kuh (2007) contend that depending on the type of qualitative assessment, more 
time might not be involved. According to the authors, good quantitative approaches to 
assessment may take just as much time, if not more time, than good qualitative approaches to 
assessment. They argue that projects which are done with more than just one person, such as 
stakeholders involved in the project, can provide more assistance to the endeavor. The added 
benefit, of course, is that the stakeholders play a large role in the project, which can insure that 
the data would be used during the implementation. It also means that the data analysis and 
interpretation could (and should) apply to multiple contexts. 

Third, certain types of qualitative approaches to assessment also can ground 
quantitative types of assessments (Donmoyer, 2012). For example, in survey construction, 
qualitative modes of knowing may provide a way to help make sure that the questions we 
are asking on a survey are not constructed in such a way that they do not make sense to our 
potential audiences. Also, if we do not know much about a topic, qualitative modes of knowing 
provide the space for an exploration into the topic or phenomena, rather than assuming we 
have intimate knowledge in regards to a topic in which we may have little or no literature or 
knowledge basis (Dillman, Smith, & Christian, 2008; Fowler, 2009). 

Finally, qualitative approaches to assessment can additionally provide a space for those 
populations who might not meet our minimum sample parameters in quantitative assessments 
to have a voice in co-constructing a narrative about student learning. As certain groups of 
students increase on campus from a population growth perspective, it is important to know 
how these emerging groups of students learn—and how their learning may be different than 
the larger groups of students on campus. Historically marginalized and emerging populations 
on college campuses might not show up in large numbers in our quantitative assessments. 
Qualitative research would teach us that this might be a time to utilize a purposive sample, 
seeking cases who are rich with information (Patton, 1990). Using a qualitative frame allows us 
to explore how these students learn on campus through more pointed and focused qualitative 
approaches to assessment, rather than quantitative modes of assessment which may exclude 
them from the analysis (such as survey data analysis depending on the statistical procedure). 

Thinking about Direct and Indirect Measures of Student 
Learning Differently 

In the narrative around assessment, there is often a distinction made between indirect 
and direct measures of student learning. Direct measures of student learning, according to 
Suskie (2009), purport to measure what students know and measure them in a fashion that 
allows for this demonstration of learning. Indirect measures seek to understand what students 
report they learned, as well as how and why the students gain knowledge in the areas being 
measured. While more and more institutions are reporting using direct measures, such as 
rubrics, to measure student learning, the use of surveys, particularly national surveys used for 



assessment purposes, still is the most prevalent form of assessment at colleges and universities 
across the United States (Kuh et al., 2014). This prevalence of indirect measures may be 
limiting what we know about student learning. 

According to Suskie (2009), direct measures should seek to measure feedback in an 
objective fashion. However, qualitative approaches to assessment which honor the tradition 
of qualitative research may have difficulty with this statement, especially in certain forms of 
direct measures of student learning. In some direct measures of student learning, an objective 
approach could actually be limiting, as we might ignore a specific population’s interpretation of 
how their learning is occurring if we do not allow room for subjective exploration. Some authors 
argue as well that objectivity is not possible in research (or expanded here to assessment— 
both quantitative and qualitative; Janesick, 2000) and instead, we should be upfront about 
how we interpreted the data, in order to be clear to readers about the framework used to 
approach the project. From the qualitative perspective specifically, Jones et al. (2014) argue 
that not being clear about how the data analyst approaches the project may actually make the 
results less trustworthy and therefore less valid. In this sense, a qualitative approach grounded 
in the foundations of research might have much to add to qualitative approaches to assessment 
in the form of thinking about validity differently. 

Traditionally, qualitative approaches to assessment have been thought of in terms 
of focus groups and interviews, but it can be much more, such as “observations, document 
analyses, and reflective journaling” (Harper & Kuh, 2007, p. 11). These three examples might 
lend themselves more towards direct measures than the traditional indirect measure of learning 
of focus groups or interviews. Additionally, when we add the layer of a “sound theoretical base” 
(Keeling et al., 2008, p. 47), we can begin to add depth to our interpretation of the qualitative 
assessment data in responsible ways. As Greswell (2003) states, the interpretation of data 
should respect the theoretical underpinnings of the chosen methods. Using a qualitative 
approach to assessment could honor this by linking to the philosophical underpinnings of the 
qualitative inquiry tradition. 

In addition, using the framework of direct versus indirect measures might allow us to 
think about focus groups and interviews in different ways as well. What might a direct measure 
of student learning, held using an interview method (for example) of data collection for the 
purposes of qualitative assessment, look like? A learning measure of this nature might appear 
as the following. A student is asked to complete a training module about a specific type of 
content. The student could then be asked to illustrate their learning about the content through 
a response to a number of scenarios which would apply the learning, and their responses could 
be evaluated via a rubric that makes clear delineations among multiple categories representing 
integration of knowledge, as well as the display of this learning. As the student answers the 
questions, the evaluator can determine where they might fit into the rubric that would best 
represent their learning. 

However, a qualitative approach to assessment adds an additional layer. Not only can 
we learn where the student might appear on the rubric, we are fortunate enough to receive 
an account as to why they are in that place on the rubric, depending on how this interview is 
conducted. We might say, “Tell me about how you would apply your learning to this situation, 
and tell me what you would do?” This is where qualitative approaches to assessment have a 
great deal of power, as we can learn more about the categories of our rubrics and how different 
types of students might arrive at our categories in very different ways. The “sound theoretical 
base” (Keeling et al., 2008, p. 47) underlying the interviews, can and should help guide our 
interpretation of the data, as well. In sum, qualitative approaches to assessment, again, allow 
us to learn more about how and what we are learning from the student perspective, and allows 
greater depth and explanatory power from our direct measure rubric alone. We can begin to 
learn more about learning. 
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Conclusion 

Qualitative approaches to assessment add a unique, and in some contexts, more 
in-depth way to answer the call for increased accountability in higher education today by 
adding a potential for deeper understanding that we may not be able to achieve through 
quantitative modes of assessment alone. By layering the frameworks that qualitative research 
provides us onto the indirect and direct measures of student learning that we seek to obtain 
from our students, we are learning from the tradition of qualitative research in such a way 
that respects this tradition by keeping the foundations of qualitative research in mind. By 
considering research and assessment as connected processes we begin to gain a way to assess 
the implementations that may come about through our engagement with research that is 
qualitative in higher education, be it in the literature or in our local contexts. We are also 
learning from those students who are participating in our institutions of higher education 
in more depth, and may be positioned to describe this learning in very rich ways as a result. 
Qualitative approaches to assessment allow us a way to learn in a more intimate way about the 
learning that is occurring at our institutions from the tremendously varied ways in which our 
students experience our universities. 


AUTHOR’S NOTE : The author would like to acknowledge the generous giving of time and 
helpful feedback from Patti Lather and Erin Castro for their assistance with this manuscript. 
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